Randomized Receiver Initiated Load-balancing Algorithms for Tree-shaped Computations

This paper considers generic load-balancing algorithms which efficiently parallelize a large class of applications based on traversing implicitly defined trees with irregular shape. First, a previous model is generalized yielding tree-shaped computations which cover the cost for communication and problem splitting, a measure of granularity and an easy to quantify parameter which limits irregularity. Then the random polling load-balancing algorithm is analyzed yielding upper bounds which match lower bounds for a large class of possible algorithms and machines. These bounds even hold for a fully asynchronous communication model which is important for practically efficient implementations. Then, with poll-and-shuffle, an asymptotically even more efficient algorithm is introduced. By using predominantly local communications, it increases the usable communication bandwidth on hypercubic networks and meshes by a logarithmic factor. These analytic results are complemented by practical refinements and implementation results which successfully apply a portable and reusable library on machines with up to 1024 processors.